docs: explain first-iteration training skip - #394
Conversation
|
Warning Review limit reachedNext included review available in 59 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Document initial-model reuse and assert that no DeePMD command runs when iteration zero has no newly labeled data. Coding-Agent: Codex Codex-Version: codex-cli 0.149.1 Model: gpt-5.6-sol Reasoning-Effort: xhigh
f88c199 to
2a95f27
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #394 +/- ##
=======================================
Coverage 84.43% 84.43%
=======================================
Files 104 104
Lines 6110 6110
=======================================
Hits 5159 5159
Misses 951 951 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
wanghan-iapcm
left a comment
There was a problem hiding this comment.
Useful bookkeeping to pick up - the skip itself shipped in dc21dfe (#116, 2023-01-25), and #73 stayed open only because that PR used no closing keyword. I checked the mechanism end to end and the core of what you wrote is right: the skip branch does write the training script, does write a skip message to train.log, and does return init_model unchanged so the supplied models flow straight into exploration.
Two things to fix.
1. The new paragraph describes a predicate master has already superseded. Your branch is based on 6b01f29, where skip_training is (init_model is not None) and (iter_data is None or len(iter_data) == 0). Master took 7485ae7 ("fix: skip training with no expanded systems", #377) on 2026-09-02, which added a second trigger:
no_iter_data = iter_data is None or len(iter_data) == 0
if (init_model is not None) and (no_iter_data or training_systems_empty):So the skip is no longer an iteration-zero-only phenomenon - it also fires in a later iteration whose configured inputs expand to zero training systems, and the log message now interpolates a reason instead of the fixed text. #377's own regression test test_exec_v2_fully_empty_training_systems builds exactly the case your sentence says cannot happen. "skips the iteration-zero training command" and "Training resumes after labeling produces iteration data" both need rewording after a rebase. The branch merges cleanly on text, so nothing forces this to surface - please rebase and re-read the paragraph against master's predicate.
While you are rewriting that paragraph, three more things belong in the same pass. They did not meet my bar to raise separately, but one rewrite fixes all of them:
init_models_urireaches the identical skip and in fact takes precedence overinit_models_pathsinsubmit.py; the dp-diststudent_model_uri/student_model_pathchain converges on the sameinit_modelsartifact too. Attributing the skip toinit_models_pathsalone documents one of three entry points, anddoc_init_models_uriis left at the bare "The URI of initial models".- "Training resumes after labeling produces iteration data" is true but incomplete in the way that matters operationally:
init_model_policydefaults to"no", so iteration 1 trains from scratch and the supplied models are used for exactly one round of exploration unless the user also sets that policy. - "those models were already trained on the initial dataset" reads as a second conjunct of a condition DPGEN2 evaluates. It is not - nothing checks provenance. The neighbouring "one model per committee member" clause is enforced (
submit.pyraises when the count differs fromnumb_models), which makes presenting the two identically more misleading than it looks. Phrasing it as a user obligation rather than a machine-tested condition would fix it.
2. do_finetune is documented in the wrong section, and the mistake it invites is silent. See the inline comment.
3. The new test assertion never executes in a regression. See the inline comment.
One thing I checked and am explicitly not raising, so it does not get re-opened: the four-sentence paragraph is not a style violation. The n_sample paragraph in the same file - blamed to 7794611 (#175), the commit that established this section's prose - is also four sentences with a worked example. Multi-sentence entries are house style when the argument needs them.
| The `"type" : "dp"` tell the traning method is {dargs:argument}`"dp" <train>`, i.e. calling [DeePMD-kit](https://github.com/deepmodeling/deepmd-kit) to train DP models. | ||
| The `"config"` key defines the training configs, see {ref}`the full documentation<train[dp]/config>`. | ||
| The {dargs:argument}`"template_script" <train[dp]/template_script>` provides the template training script in `json` format. | ||
| When {dargs:argument}`"init_models_paths" <train[dp]/init_models_paths>` supplies one model per committee member and those models were already trained on the initial dataset, DPGEN2 automatically skips the iteration-zero training command. It records the generated training script and a skip message, then passes the supplied models directly to exploration. Training resumes after labeling produces iteration data. Finetuning requested with `"do_finetune": true` is never skipped. |
There was a problem hiding this comment.
"do_finetune" is an inputs argument, not a train one. It is declared in input_args() and read as config["inputs"].get("do_finetune", False) in submit.py. This sentence sits in ### Training, directly under the "train": {...} block, and - unlike "config", "template_script" and "init_models_paths" in the same paragraph - carries no {dargs:argument} role, so a reader has no link to follow to the right section.
The mistake that invites is silent. normalize() calls check_value(..., strict=False), so I put "do_finetune": true under "train" and normalization succeeded, the key was retained verbatim under train, and inputs.do_finetune stayed at its False default. No error, no finetuning - and therefore the iteration-zero skip that this very sentence promises would not happen.
{dargs:argument}`"do_finetune" <inputs/do_finetune>` would both fix the link and make the section obvious.
| jdata = json.load(fp) | ||
| self.assertDictEqual(jdata, self.expected_odict_v2) | ||
| self.assertEqual(Path(out["model"]).read_text(), "this is init model") | ||
| mocked_run.assert_not_called() |
There was a problem hiding this comment.
This assertion cannot execute in any scenario where it would fail, so it does not deliver the PR body's "assert the skip path never invokes a DeePMD training or freeze command".
I injected return False at the top of RunDPTrain.skip_training so the skip never fires. The test does fail - but at ret, out, err = run_command(command) with ValueError: not enough values to unpack (expected 3, got 0), because a bare MagicMock yields nothing. Line 993 is never reached. Adding mocked_run.return_value = (0, "", "") moves the failure earlier still, to the pre-existing assertEqual on out["model"]. The reason generalises: the skip branch returns init_model while the training branch always names a different path, so no state exists where run_command was called and the preceding assertions passed.
Running the same mutation against the pre-PR version of this test produces an identical failure set, via FileNotFoundError: No such file or directory: 'dp' - so the new line adds no detection power.
The @patch decorator itself is a real improvement and worth keeping: deepmd-kit is not in the test extra, so dp is absent in CI, but on a developer machine that has it installed the pre-PR test would have shelled out to a real dp train.
Master's sibling test test_exec_v2_fully_empty_training_systems shows the working pattern - it sets mocked_run.side_effect = [(0, "foo\n", ""), (0, "bar\n", "")] before execute() and asserts immediately after, which is why its own assert_not_called() reports properly ("Expected 'run_command' to not have been called. Called 2 times.").
Summary
Tests
Closes #73
Coding agent: Codex
Codex version: codex-cli 0.149.0
Model: gpt-5.6-sol
Reasoning effort: xhigh